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ABSTRACT 

Let  be  a  sequence  of  solutions  to  an  ordinary 

differential  equation  with  random  right  sides  (due  to  input 

noise  (C  (•)})  and  which  converges  weakly  to  a  diffusion  x(0 

with  unique  invariant  measure  Let  u(t,*)  denote  the 

measure  of  x(t),  and  suppose  that  y(t,*)'*’  Vi(*)  weakly.  The 

paper  shows,  under  reasonable  conditions, that  the  measures  of 
£ 

X  (t)  are  close  to  vi(*)  for  large  t  and  small  e.  in  ap¬ 
plications,  such  information  is  often  more  useful  than  the  simple 

fact  of  the  weak  convergence.  The  noise  5^(0  need  not  be 

£ 

bounded,  the  pair  (x  (•)»  C(*))  need  not  be  Markovian  (except 

for  the  unbounded  noise  case),  and  the  dynamical  terms  need  not 
be  smooth.  The  discrete  parameter  case  is  treated,  and  several 
examples  arising  in  control  and  communication  theory  are  given. 

This  research  was  supported  in  part  by  the  Air  Force  Office  of 
Scientific  Research  under  AFOSR- 76- 3063®,  in  part  by  the  National 
Science  Foundation  under  NSF-Eng.  ,77-12946-A02 ,  and  in  part  by 
the  Office  of  Naval  Research  under  \’00014- 76-C-0279-P0004 . 

\ 


I.  INTRODUCTION 


Many  results  are  available  concerning  the  weak  convergence 
of  a  sequence  of  processes  {x  C*))  to  a  diffusion  x(»)  Cwith 
values  in  ,  where  are  the  solutions  to  ordinary 

differential  equations  with  random  right  hand  sides  [1]-[5J; 
e.g.,  where  ±  =  FgCx  ,5  )  for  some  function  Fg.C.»,*)  and  a 

"wide  band"  noise  process  For  small  e  >  0,  the  weak 

convergence  basically  gives  us  information  on  the  approximation 
of  x^C*)  by  x(0  on  arbitrarily  large  but  still  finite  time 
intervals.  In  applications  to  control  and  communication  theory, 
information  concerning  the  asymptotic  behavior  -  for  small  e, 
but  for  "large"  t  is  often  of  much  greater  interest.  For  ex¬ 
ample,  we  want  to  know  how  well  the  asymptotic  distributions 
(the  distributions  at  arbitrarily  large  times)  of  the  x  (•) 
are  approximated  by  the  (say)  invariant  measure  of  x(0  for 
small  E. 

This  problem  was  discussed  in  [2,  Section  6]  and,  roughly 

speaking,  a  result  of  the  following  type  was  obtained.  Let  ^(•) 

£ 

be  a  stationary  finite  state  jump  Markov  process,  define  5  (t)  = 
2 

^(t/e  )  and  for  smooth  functions  F(*,*)»  G(*,*)»  G(*),  where 
EF(x,^i(t))  H  0  s  EG(x,5(t)),  define  x^(*)  by 

(1.1)  =  F(x,?;^)/e  +  G(x,^^)  +  G(x),  x^t)  €R^  . 


£  £ 

Then,  if  x  (0)  converges  weakly  to  x(0),  {x  (•))  converges 

weakly  to  a  diffusion  x(*)  whose  generator  is  defined  by 


(1.2)  =  f;(x)G(x)  + 


E(r  (x)F(x,5(t)));F(x,5(0))dt 


0 
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Suppose  that  x(0  has  a  unique  invariant  measure 
and  let  there  be  a  smooth  Liapunov  function  0  £  VCx)  ■*■'*>  as 
lx|  and  a  Y  >  0  such  that 

(1.3)  isi^VCx)  <  -  YV(x),  for  large  |xl  . 

£  £ 

Then,  for  small  e,  (x  (•),  5  (•))  has  an  invariant  measure 
£  £ 

V  (•)  whose  x-marginals  y  (•)  converge  weakly  to  y(*)  as 
e  ->■  0  [2 ,  Section  6]  . 

We  are  interested  in  results  of  the  same  type,  but  where 

£ 

5  (•)  might  not  be  Markov  -  or  even  bounded  (Gaussian,  for 

example),  and  where  F(*,*)  and  G(*,*)  might  not  be  smooth. 

Also,  in  many  interesting  applications  (1.3)  does  not  hold  (e.g., 

in  many  cases  where  F(*,«)»  G(«,*)  and  G(»)  are  bounded). 

£  £ 

Thus,  for  our  problem  (x  (•),  5  (•))  might  not  have  an  invari¬ 
ant  measure,  even  for  small  e  >  0.  Furthermore,  we  are  also 

£ 

interested  in  the  case  where  the  evolution  of  5  (•)  depends  on 

e 

X  (•).  Also,  [2]  does  not  address  the  question  of  the  distribu- 
£ 

tions  of  X  (t)  being  close  to  u(*)  for  large  t  and  small 
e  and  arbitrary  initial  condition,  a  problem  which  is  important 
in  applications.  The  basic  techniques  used  here  are  similar  to 
those  in  [2];  both  heavily  depending  on  the  use  of  "averaged 
Liapunov  functions." 

Section  2  contains  the  basic  approximation  theorem,  but  using 
the  condition  (AS)  which  is  not  usually  directly  verifiable.  A 
verifiable  condition  for  (AS)  appears  in  Section  4. 

Since  our  interest  is  only  in  the  asymptotic  properties  of  the 


measures  of  the  {x  C*)/  for  small  e,  we  assume  weak  convergence 
(A4)  -  preferring  to  keep  close  to  the  main  purpose.  In  any  case, 
many  known  sets  of  conditions  imply  the  weak  convergence  (see 
[l]-[5]  or  the  conditions  in  Section  4).  Section  4  also  discusses 
the  changes  required  when  F(*,*)  or  G(*,0  are  not  smooth,  a 
case  which  is  important  in  many  applications.  The  "unbounded" 
noise  case  is  dealt  with  in  Section  5,  some  examples  are  dis¬ 
cussed  in  Section  6,  and  in  Section  7  we  treat  the  discrete  param¬ 
eter  case,  when  the  noise  might  be  "state"  dependent.  Section  3 
contains  a  result  that  is  useful  in  the  proof  of  Section  4. 

For  future  reference,  we  note  that  if  5(*)  is  Markov,  it 
is  sometimes  convenient  to  write  (1.2)  in  the  form 

(1.4)  y>f(x)  =  f^(x)G(x)  +  j  E(fj^(x)E^^O^F(x,C(t)))/(x,C(0))ds 


The  advantage  of  (1.4)  is  that  if  F(*,*)  is  not  differentiable, 
then  the  derivative  in  (1.2)  might  not  exist,  but  that  in  (1.4) 
might  exist  for  t  >  0,  and  we  can  weaken  the  conditions  required 
on  F(*,*),  and  even  on  G (•,•)♦ 


-4- 


2.  THE  BASIC  CONVERGENCE  THEOREM 


Assumptions 

A1 .  X ( • )  is  a  Feller  diffusion  process  with  continuous  coef¬ 
ficients  and  a  unique  weak  sense  solution  on  [0,“)  for 
each  intial  condition. 

A2.  x(*)  has  a  unique  invariant  measure  uC*)  and  PCXjt,*). 

the  measure  of  x(t)  when  x(0)  =  x,  converges  weakly 
to  for  each  x,  ^  t 

A3.  The  convergence  in  (A2)  is  uniform  in  compact  x-sets;  i.e., 
for  each  f(0  €  the  space  of  continuous  bounded 

functions  on  R’^,  E^f(x(t))  E^f(x(0))  uniformly  in  x 

in  compact  sets,  where  E^  denotes  expectation  with  respect 
to  the  stationary  measure  of  x(0*  ((A3)  need  only  hold  for 

the  functions  of  interest  in  the  Theorem.) 

A4.  X  (•)  -*■  x(*)  weakly  (initial  condition  x(0))  ~~if  x^(0) 
x(0)  weakly  in  D^[0,®). 

A5.  There  is  an  >  o  such  that  {x^(t),  0  <  e  £  t  ^  0} 

is  tight.  (See  Theorem  3  for  a  verifiable  criterion  for  (AS).) 

Remark  on  (A3).  Condition  (A 3)  is  implied  if  il  is  weakly 
stable  in  the  sense  of  Liapunov.  More  specifically,  let 
be  a  sequence  of  continuous  and  uniformly  bounded  functions,  each 
with  compact  support  such  that  Uq 


weakly  (as  ot  -►  O)  is 
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equivalent  to 
u(t,0  denote 
6  >  0  and  f 


the  distribution 

,  ....  f  ,  there 
m 


of  xCt).  Suppose  that  for  each 
are  e  >  0  and  n  <  o’  such 


that 


fi(y)w(o.dy) 


fi(y)u(dy) 


<  e , 


i  <  n 


implies 


fi(y)u(t,dy) 


f^(y)u(dy) 


<  6 ,  i  £  m, 


all  t  >  0 


Then  (A4)  holds.  In  the  completely  degenerate  case,  where  xC*) 
satisfies  an  ordinary  differential  equation  and  u  is  concentrated 
at  a  point  Xg,  then  the  above  criterion  is  equivalent  to  the 
usual  Liapunov  stability  of  Xg. 


Lemma  1.  For  any  integer  m,  let  f  (•)£  Assume  CA1)-(A3) 

and  let  0  =  <  A,  .  .  .  <  A  .  Let  S  denote  a  tight  set  of 

-  0  1  m  -  - - 

valued  random  variables.  Then 

^xC0)f^x(t  +  A^),  i  <  m)  E^f(x(A^),  i  <  m) 

uniformly  for  x(0)e  S,  ^  t 

The  result  is  a  consequence  of  CA1)-CA3),  and  the  proof  is 
omitted.  The  uniformity,  in  particular,  requires  CA3) .  The  A^ 
in  Theorem  1  are  the  same  as  those  in  Lemma  1. 

Theorem  1.  Assume  CA13-(A5).  Then  for  each  fCO^  -^m+r 


- 


-6- 


<S  >  0,  there  are  tpCf.i)  <  «  ^  eo(f,6)  >  0  such  that  for 

^iA  ^  i  ^  and  any  sequence  {x^(*)^ 

which  converges  weakly  to  x(*)  , 

(2.1)  |Ef(x^(t  +  A^) ,  i  <  m)  -  Eyf (x(A^) ,  i  <  m) j  <  6  . 

^(O  be  Markov  and  (x  (*),C  (•))  Markov  and  have  an  invari¬ 
ant  measure  v'^CO.  Replace  (AS)  bj^:  There  is  a  sequence  » 

depend  on  the  initial  condition)  such  that  (A5)  holds 

^  Then  {v^(«),  small  e}  is  tight  and  its  x-marginals 

(y  (•)}  converge  weakly  to  y(»)* 


Remark.  The  theorem  implies  that  the  convergence  as  t  ->■  <*>  in 

(2.1)  is  uniform  in  e  for  small  e,  a  fact  which  is  important 

in  applications.  In  applications,  it  is  often  possible  to  prove 

results  such  as  lim  Ct)l^  K,  where  K  does  not  depend  on 

^->■00 

.  c 

e  or  X .  Then  if  (C  C*))  is  bounded,  the  replacement  for  (A5) 
in  the  last  paragraph  holds. 


Proof. 

e  -►  0 


Suppose  that  (2.1)  is  false.  Then  there  is  a  subsequence 
and  a  sequence  {t^}  ^  »  guch  that 


(2.2) 


|Ef  (x^(t£-^  A.)  ,  i  <  m) 


Ejjf  (x(A.)  ,i  <  ra)  I  >  a  >  0 


“  /  - 


We  will  find  a  further  subsequence,  also  indexed  by  e,  which  vio¬ 
lates  C2.2).  Fix  T  >  0.  By  (AS),  we  can  choose  a  further  sub¬ 
sequence  such  that  {x^(t^-  T) }  converges  weakly  to  a  random 
variable  x(0).  By  (A4) ,  T  +  •)>  converges  weakly  to 

x(0  with  initial  condition  x(0)  and 

(2.3)  EfCx^Ctg,-  T  +  T  +  A^),  i  <  m)  EE^^Q^f(x(T  +  A^)  ,  i  <  m)  . 


By  (AS),  the  set  S  of  all  possible  x(0)  (over  all  T  >  C  and 
weakly  convergent  subsequences)  is  tight.  By  Lemma  1,  we  can  take 
T  large  enough  such  that 

(2.4)  |EE^^Q^f(x(T  +  A. )  ,  i  <  m)  -  Ej^f(x(A.),  i  <  m)  !  <  6/2. 

Equations  (2.3)  and  (2.4)  contradict  (2.2). 

The  proof  of  the  last  assertion  is  similar  to  the  last  part  of 
the  proof  of  Theorem  4  and  is  omitted. 

Q.E.D. 


.J 


-Jl— 


I 


I 


i: 
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3.  A  LIAPUNOV  FUNCTION  CRITERION  FOR  (3.1) 

(3.1)  U  (xCt),  t  >  0  ,  x(0)  =  x}  is  tight  for  each  compact  B. 
xeB 

To  prove  (3.1),  we  will  require  condition  (A6) . 


A6.  There  is  a  continuous  Liapunov  function  0  ^  VCx)  ^ 

|x|  -*■  ”  and  a  and  ag  >  0  such  that  V(:x)  5.  -  ^^g 

for  X)?  Qg  =  {x:V(x)  £  ^0^*  The  partial  derivatives  of  V(*) 
up  to  order  2  are  continuous. 

Theorem  2  is  proved  partially  because  the  proof  is  a  prototype 
of  the  technique  used  later  to  verify  (AS) . 


Theorem  2.  Under  (Al)  and  (A6),  condition  (3.1)  holds . 

Proof.  (A6)  implies  that  Qg  is  a  recurrence  set  for  x(‘)  [6]. 
We  suppose  w.l.o.g.  that  min  V(x)  =  0. 

X 

Let  X2  >  ^  ^0  define  =  {x:V(x)  ^  X^}.  Let  tg 

denote  a  Markov  time  such  that  ^  Qg»  and  define  = 

min{t:t  >  Tg,  x(t)  €  Qg}-  Then  ([6])  an  application  of  Ito's  for¬ 
mula  yields  that  for  any  Markov  time  t  ^  tg  for  which 

■  ■‘o>  ' "  ^ 


(3.2)  )VCxCt  n  Tj))  -  V(X(V)  < 


tnt, 
f  1 


'i^V(x(s))ds 


0 


i 


i 

'A 

1 


(3.3) 


E 


V(x(To))/ao 


X(t 


>}  =  1 
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(3.4)  sup  V(x(s))  >  X}  £  V(x(Tp))/X  . 

0-' 

Define  Tq  = 

To  get  (3.1),  define  a  sequence  of  Markov  times  as 

follows.  For  n  =  1, 

=  min{t:x(t)  £  Qq)  . 

For  n  >  1, 

°n  '  °n-l*  ■'o  “  ^  ‘^1  " 

0  -  lnf{t:t  >  x(t)€  3Qj)  if  qJ  but 

x(t)€  3Q^  for  some  Tq] 

=  inf{t:t  >  x(t)C  Qq)  if  x(a^_^)^  Q° 

For  n  >  1,  E  ,,  o„)  i  T^,  and  <  -)  =.l.  In  fact 

if  the  set  B  in  (3.1)  is  contained  in  Q^,  then  sup  ^ 

xCB 

by  (3.3).  Fix  6  >  0.  For  each  k  and  t  define  j (t,k)  by 
‘’j(t,k)’  i  Vi’  i  • 

Define  the  intervals  Aj  =  iPj  (t  ,k)*  (i- 1)  ’  °j(t,k).i’'  ^  ^ . 

Then  for  ^  1 

k 

(3.5)  P^{V(x(t))  >  A}  <  Px^<?j(t,k)  -  *-1^  ^^^seA. 


+  6/4. 


We  can  choose  k  such  that  the  first  term  on  the  right  hand  side 


is  <  6/4  for  all  x£B.  Then,  by  (3.4),  we  can  choose  X 
that  the  sum  is  <  6/4  for  all  x.  Since  V(x)  -*■  «  as 
(3.1)  holds. 


such 
1x1  ~, 

Q. E. D. 
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4 .  An  Averaged  Liapunov  Function  Criterion  for  (A5) . 

In  this  section,  we  use  the  model  (1.1)  and  a  strong  mixing 
condition  on  C(*).  The  development  should  be  viewed  as  an  il¬ 
lustration  of  a  general  technique.  The  mixing  condition  is  too 
strong  for  many  applications,  and  other  conditions  are  consid¬ 
ered  in  Section  5.  The  mixing  condition  is  used  simply  to  assure 
certain  bounds.  In  specific  cases,  a  very  similar  development 
can  be  carried  through  under  other  conditions  on  the  noise,  and 
the  same  bounds  shown.  Also,  (see,  e.g..  Example  2  in  Section  6), 
a  very  similar  development  can  often  be  carried  through  for  equa¬ 
tions  of  forms  other  than  (1.1).  The  smoothness  requirement  in 
Condition  (B2)  is  weakened  in  the  remarks  after  the  proof.  In 
order  to  get  the  necessary  inequalities  for  any  Liapunov  function 
based  approach,  an  assumption  such  as  (B4)  seems  to  be  required. 
The  conditions  hold  in  numerous  cases  of  interest. 

We  will  use 

B1 .  ^(O  is  a  bounded,  right  continuous,  stationary  <}> -mixing 

~  00 

process  [7]  with  |  <{>  (t)dt  <  ~  . 

0 

B2.  F(*,*),  G(*,*)  and  G(*)  are  continuous,  valued 

functions  whose  growth  (as  |xl  -*■'»)  ^  O(lxj).  The 
partial  derivatives  of  F(*,C)  up  to  order  2  (and  of  G(*,^) 


up  to  order  1)  are  bounded  uniformly  in  x,C,  and  EF(x,5) 
H  0  H  EG(x,C)  . 


%— 
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B3.  There  is  a  diffusion  process  x(0  with  differential  genera 
tor  ^  defined  by  (1.2),  and  which  satisfies  CA1)-(A3). 
Also,  (A6)  holds, .but  the  partial  derivatives  of  V(*)  up 
to  order  3  are  continuous. 

B4.  There  are  constants  K  such  that,  uniformly  in  x,S, 

(4.1a)  |V^(x)G(x,5)  1  +  |V^Cx)F(x,5)  !  <  (1  +  V(x)) 

(4.1b)  I  (V^(x)F(x,5))^  F(x,C)!  1  K(1  +  V(x)) 

(4.2)  I  (V^(x)G(x,5))^  U(x,5)|  1  K(1  +  |i^V(x)I) 

for  U  =  F,G,G 

(4.3)  1  (V^(x)F(x,^))^  U(x,^)l  <  K(1  +  li*-V(x)l),  U  =  G,G 

(4.4)  !  (0^3^Cx)F(x,C));  F(x,^))’  U(x,5)  I  <  K(1  +  !.^V(x)I)  , 

U  =  F,G,G  . 

Note  that  here  and  in  the  sequel  the  value  of  K  might  change 
from  usage  to  usage. 

''  e  -V  ''  e 

Define  the  differential  operator  A  and  its  domain  ) 

as  in  [4],  [5],  [8].  The  method  of  use  of  A^  is  similar  to  that  in 

[4]  and  the  averaging  method  is  similar  to  that  in  [2],  [4],  [5]. 

Theorem  3.  Under  (B1)-(B4)  and  the  tightness  of  {x^(0)},  condi- 
(A5)  holds. 
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Proof.  For  each  integer  N,  define  !^!  £  N}.  Let 

bj^CO  denote  a  function  with  values  in  [0,1]  and  satisfying 
bj^(x)  =  1  in  Sj^  and  equals  0  in  R  -  and  th.e  partial 

derivatives  up  to  order  3  are  bounded  uniformly  in  N,x.  Define 
Vj^CO  =  V(*)bj^CO-  The  V(*)  is  "truncated"  because  we  cannot 
apply  to  unbounded  functions  without  additional  conditions. 

V(0  is  the  Liapunov  function  which  is  to  be  "averaged,"  and 

plays  the  role  of  a  "differential"  generator  for  x  CO*  Let 

£  £  £ 

denote  expectation  conditioned  on  5  (s) ,  s  ^  t,  x  (0).  We 

£ 

have  (writing  x  =  x  (•)) 


e 

C4.5)  A^Tj^Cx)  =  V’^^(x)  +  G(x,5^(t))  +  G(x)^  . 

To  average  out  Vj^j^^Cx)  G(x,5^(t)),  define  V^^p(t)  =  V^,^Q(x^(t)  ,t) , 
where 

00 

=  j  G(x,5^(t  +  s))ds  . 

0 

2 

By  changing  variables  s/e  ->  s  and  using  (Bl)  and  (4.1),  we  get 
(the  K  do  not  depend  on  N) 


(4.6)  IVj^Q(x,t)l  <  e  K(1  +  V(x)),  xGSj^  . 

We  have  q(*)  (A^)  and  (write  x  =  x^(t)) 

A^V;^  0(x,t)  =  -  ^(x)  G(x,5^(t)) 

00  ®  ^ 

^  jds  G(x,5^(t  +  s))}^{g(x)  +  G(x,5^t))  + 
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2 

By  changing  variables  s/e  ^  s  and  using  (Bl)  and  (4.2),  we  get 
that  the  integral  is  bounded  by 


(4.8) 


e  K(1  +  |W(x)  I),  x€S 


N 


We  now  proceed  to  average  out  the  V'  (x)  F(x,5  (t))/e  com- 

I N  j  Jv 

ponent  of  (4.5).  Define  j^(t)  =  2^(x^(t)»t),  where 

^t  *  s))ds/e 


VN^l(x,t) 


0 


By  (Bl)  and  (4.1),  and  the  change  of  variables  s/e^  ->■  s,  Vj^  j^(x,t) 
satisfies  the  bound  in  (4.6).  Also»  Vj^  j^(*)  £  ^(A  )  and  (write 
X  =  x^t)) 


(4.9)  A^V^^^(x,t)  =  -  Vj;j^^(x)  F(x,5^(t))/e  + 

00 

ds 

0 

*ecx)}  . 

2 

By  (Bl) ,  (4.3)  and  the  change  of  variables  s/e  -*•  s,  the  terms 
in  (4.9)  involving  G  +  G  are  bounded  by  (4.8). 

The  remaining  term  in  (4.9)  must  now  be  averaged  out.  Define 


V'  (x) 
t  N,x^  ^ 


F(x,r 

e 
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CO  00 

(4.10)  =  I  dt  I  ds  {(V^^^(x)  F(x,5^(t  +  s  +  T)))^FCx,5^(t+T)) 

’  0  0  ’ 

-  ECV'^^Cx)  F(x,C^(t  +  s+Tj))^F(x,5''(t  +  T))}/e^  . 

2  2 

By  (Bl) ,  (4.3)  and  the  change  of  variables  s/e  -►  s,  T/e  -v  x, 

bounded  by  (4.6).  Also,  2^’^  £  ^^(A^)  and  (writing 

X  =  x''(t)) 

(4.11)  “  *  (inner  integral  of  (4.10)  evaluated  at 

T  =  0)  .  . 

By  (Bl) ,  (4.4),  and  the  usual  change  of  variables,  the  last  term  on 

the  right  side  of  (4.11)  is  bounded  by  (4.8). 

2 

Define  V^(x,t)  =  Vj^(x)  +  l  and  V®(t)  =  V^(x^(t),t). 

Define  V^(x,t)  =  vj^(x,t).  For  xESj^, 

(4.12a)  !V^(x,t)  -  Vj^(x)!  <  e  K(1  +  V(x)) 

(4.12b)  A^v''(t)  =  :/Vj^(x''(t))  +  0(e)  (1  +  |^V(x^(t))|)  . 

Also,  V^(x,t)  >  -  Ke,  and  the  expression  in  (4.12b)  is  £  -  ®g/2  in 
Sj^-  Qg  for  small  e  >  0. 

£ 

Let  Tq  denote  a  stopping  time  such  that  x  (Tg)^Qg  and  de¬ 
fine 

=  inf{t:  t  >  Xp,  x^(t)€Qg}  , 

Xj^  =  inf{t:  x^(t)  e  s^}  . 
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Then,  for  small  enough  £  >  0,  and  t  any  stopping  time  satisfying 
T  >  and  ECJ  '  <  •», 


(4.13) 


T^nTj^nt 


=  E 


A^V^Cx^(s) ,s)ds 


-  '  2  CJ]^nT,^nT)  -  (x^ntj^)  ] 

£ 

The  N  subscript  on  Vj^  can  be  dropped,  since  the  x-  argument  is 
always  in  Sj^.  Now,  let  N  ->  «  (hence  ->■  since  by  (62),  there 
is  no  finite  escape  time  for  any  e  >  0).  Then  let  x  -»>  <».  Then 
(4.13)  and  the  fact  that  V^(x,t)  >  -  Ke  yield  (for  small  £) 


(4.14)  E^^(x^  -  Xq)  <  2[V^(x^Tp),Xp)  +  K£]/ap 

1  3[V(x^Xq))  +  K£]/ap  , 

and 

-  Ke  +  {  sup  V^(x^(s),s)  >  X}  <  |V^(x^(x.)  ,x  )  ) 

0  y  0 

from  which  we  get 

r  e  )  2V(x^x  ))  +  Ke 

'■"o  ijs>x„  -  ■ 

Inequality  (4.14)  implies  that  the  mean  travel  time  from  to 

£ 

3Qq  for  x  (.*)  is  bounded  by  3(Xj  +  The  proof  is 


Li 


completed  as  in  Theorem  2  with  (4.14)  and  (.4.15)  replacing  (.3.3) 
and  (3.4),  respectively. 

Q .  E .  D . 


Remark  on  Theorem  3  for  non-smooth  F  and  G.  Let  ?(•)  be 
Markov  with  transition  function  P(.§,t,r)  and  invariant  measure 
P(r).  Then  even  when  F(*,C)  and  G(*,5)  are  not  suitably  dif- 
ferentiable,  the  constructed  Vj^C*)  might  still  be  in  ^(.A  ) 
and  the  bounds  (4.6)  and  (4.8)  might  hold.  For  this  non-smooth 
case,  we  write  the  derivative  which  appears  on  the  right  side  of 
(4.7)  as 


(.4 


.16)  jj  V^^^(x)  G(x,5j)  P(C^(t),  ^  ,  d§j^)] 


This  gradient  might  exist  for  s  >  0,  even  though  GC*,5)  is  not 
smooth.  Similarly  for  the  other  terms.  In  particular  write  the 
integrand  in  (4.10)  in  the  form 


(4.17) 


f(x,C2^ 


s 


dC2) 


F(x,^)  P(5  (t) 


7’ 


J 


V;,  (x) 

N,x  ^  ^ 


f(x,52^ 


s 

77’ 


d^,) 

2'^J  X 


F(x,^^) 


P(d^j) 


£ 

In  any  case,  if  with  these  representatives,  the  constructed  VjjCO 

A  £ 

is  in  ^(A  )  and  (4.6)  and  (4.8)  hold,  then  Theorem  3  continues  to 


hold,  if  F(*,*)  and  G(*,*)  are  measurable  but  do  not  satisfy  the 
continuity  and  differentiability  assumptions  in  (B2). 
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5 .  Unbounded  Noise 

Suppose  that  (1.1)  is  used  and  that  the  5C*)  there  is  Markov, 
but  unbounded.  Then  Theorem  3  is  not  directly  valid.  But,  fre¬ 
quently  in  applications  cases  arise  which  can  be  treated  by  essen¬ 
tially  the  same  proof,  due  to  the  special  way  in  which  the  noise 
enters,  and  the  properties  of  the  dynamical  terms.  We  now  describe 
one  such  case  -  which  was  abstracted  from  the  situation  arising  in 
several  examples  when  one  tries  to  apply  the  method  of  Theorem  3. 

Many  important  examples  fit  the  situation  to  be  described.  See,  for 
example.  Example  2  of  Section  6,  which  is  typical  of  a  large  class 
of  such  cases. 

£ 

We  suppose  that  the  constructed  in  Theorem  3  are  in 

for  ^  where  >  0,  and  that  (1.3)  holds  and  that 

there  are  random  variables  5  (t) ,  5  (t) ,  integers  p,q  and  func¬ 
tions  V(*)  and  V(*)  satisfying  (5.1).  For  the  first  two  lines 

of  (5.1)  ,  let  xeSj^,  Let  v^(t,*)  denote  the  measure  of  (x^ (t)  ^ (t) ) 

£  £ 

and  let  Eg  denote  expectation  conditioned  on  (5  (0),x  (0)). 

A^V^(x,t)  +  ^(x)  +  0(e)  (1  +  V(x))  +  0(e)  5^(t)  V(x,t) 

VN(x,t)  +  V^(x)  +  0(e)  (1  +  V(x))  +  0(e)  C^t)  V(x,t) 

(5.1)  ^ 

sup  ElC^(t)|P  <  “>  ,  sup  ElC(t)|’^  <  * 
t,e  t,e 

|V(x,t)  OCVCx))  -  I^Dc.t) 


for  large  [x]  • 
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Example  2  in  Section  6  describes  an  important  class  where 
these  conditions  hold.  For  another  example,  consider  the  case 
where  ?(•)  is  Gauss-Markov  and  F(x,5)  =  Fq(x)5,  G(x,C)  =  Gq(x)5 
and  FqCx),  Gq(x)  as  bounded  and  smooth  and  G (x)  ^  -  YV(x) 

+  K,  Y  >  0. 

We  shall  need 

Cl.  G(*)f  F(*>0>  G(‘,*)  are  measurable  and  are  0(1x1)  for 
large  |xl,  uniformly _ bounded  5*sets. 

C2.  For  the  given  sequence  {x^(0)},  sup  EV(x^(0))  <  “. 

C3.  sup  El5(t)I  <  ».  For  small  £.  (x^  (• )  ^(  • ) }  and  5(0  are 
t 

Markov-Feller  processes  with  right  continuous  paths  and  homo¬ 
geneous  transition  functions. 


C4.  There  are  Y  >  0  and  Kj^<  «>  such  that  y'V(x)  ^  -  YV(x)  +  K^. 


Remark.  Condition  (C2)  facilitates  the  proof  but  is  not  necessary 
for  the  result. 

Theorem  4.  Assume  (C1)-(.C4),  (A4) ,  CB3) ,  (5.1)  and  the  conditions 
above  (5.1).  There  is  an  >  0  such  that  for  ^  1  (x^(0> 

C  C  £ 

5  CO)  has  an  invariant  measure  v  (•)  •  The  x-marginals  (y  CO) 
of  any  such  sequence  of  invariant  measure  converge  weakly  to  P (O 
as  e  -►  0.  Also  (A5)  and  the  conclusions  of  Theorem  1  hold. 
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Remark.  If  FC*»’)  Of  G(*,*)  are  not  smooth,  see  the  remark  for 
Theorem  3.  The  theorem  is  an  extension  of  the  result  in  [2,  Sec¬ 
tion  6]  in  that  both  use  CC4). 


Proof.  Define  Tq  and  as  in  Theorem  5.  By  the  hypotheses, 

for  any  positive  T, 


(5.2) 


EV^Cx^CTnXj^),  TflTj^)  -  EV^(x^(0),0) 


<  E 


Tnx 

0 

TflT 


N 


A^V^Cx^Cs)  ,s)ds 


0 


^  [ijrv(x^(s))  +  0(e)  +  0(e)V(x^(s)) 


+  0(e)lC''(s)V(x^s),s)I]ds  . 


By  (Cl),  as  N  ^  ”  we  have  »  w.p.l.  The  limit  as  N  ->■  <» 

of  the  right  side  of  (5.2)  is  bounded  above  by 


(5.3) 


EV(x  (s))  (1  +  0(e))ds 


0(e) 


^  eP'^/P  1V(x^(s) ,s) 1P^P'^E^/P!5^(s) ds  +  K_T 


<  -  Y(1  -  0(e)) 


EV(x  (s))ds  +  2K2T 


for  some  real  K 
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Similarly  the  limit  as  N  ^  of  the  right  side  of  (5.2)  is 
bounded  below  by 

(5.4)  EV(x^(T)) (1  -  0(e))  - 

for  some  real  K^.  Thus  for  some  >  0 , 

T 

(5.5)  EV(x^(T))  <  K^-  Y,  f  EV(x^(s))ds 

J  0 

+  2K2T. 

By  (5.5),  EV(x^(T)),  T  >  0,  e  <  some  small  e^}  is 

bounded  by 

e 

(5.6)  EV(x  (T))  1  K^e  ' 

This  and  (C3)  and  a  Theorem  of  Benes  [11]  imply  that  for  small  e 

C  £  c 

there  is  an  invariant  measure  v  (•)  for  (x  (0»5  (•))  and  the 

first  assertion  of  the  Theorem  is  proved.  By  (5.6),  the  x-marg- 

inals  {y^(*)>  of  {v^(*)}  are  tight.  By  this  and  (C3)  so  is 
£ 

{v  (•)}■  Also,  by  (5.6)  and  the  pronerties  of  V(>)> 

{x^(t),  e  £  ,  t  ^  0}  is  tight.  Hence  (A5)  and  the  conclusions 

of  Theorem  1  hold. 

Now,  we  prove  the  middle  assertion  of  the  Theorem.  Let  v  (•) 

G  £ 

be  the  measure  of  (x  (0),  5  (0)),  and  let  x(0  denote 

£ 

the  weak  limit  of  any  weakly  convergent  subsequence  {x  (•))•  By 

e  £  £ 

the  invariance  of  v  (•),  the  distribution  of  x  (t)  is  y  (t) 

for  each  t.  Thus,  x(t)  has  the  same  distribution  for  each  t. 


-X, 
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This  must  be  u(*)>  by  the  uniqueness  of  the  invariant  measure 

£ 

U(*)-  In  fact,  if  denotes  any  sequence  of  invariant 

measures  for  {x  CO»5  (*)!  for  which  the  x-marginals 
are  tight,  then  we  get  the  same  result. 

We  complete  the  proof  by  showing  that  any  sequence 
small  e}  of  invariant  measures  must  be  tight.  Suppose  that. 

{v^(*)}  is  not  tight.  Then  for  some  sequence  e  ->■  0  there 
are  -»•  <»  and  a  6  >  0  and  (for  each  e  in  the  sequence) 

a  sequence  ^  ^  ns  n  ->■  “  and  an  x^.  for  which 

(5.7)  P{|x^(t^)  I  >  N^|x^(0)  =x^}  >  6 

£  £ 

(The  measures  of  C  (t)  are  all  the  same,  since  C..  (0) 

=  5(0),  and  the  5 -marginals  of  v^(0  are  all  the 

same.)  But  (5.6)  implies  that  lim  E  [V  (x^  (T) )  |  x^  (0)  =  x^.]  £  2K2/^2» 

a  bound  which  depends  neither  on  x^  or  on  e.  This  contradicts 

the  assertions  that  N_  ->  <»  as  e  ->■  0  and  -*■  “  as  n  -*■  «> 

^  n  "  ’ 

Thus  (v^C*)}  must  be  tight. 


Q.E.D. 


F 


■  x_. 
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6 .  Examples 

The  two  examples  below  arise  in  applications  to  control  and 
communication  theory.  They  were  chosen  to  illustrate  the  general 
results  and  applicability  of  the  methods  of  Theorems  1,2,4  under 
slightly  different  conditions. 

1.  Adaptive  antenna  arrays.  Let  vC*)  denote  a  complex  R 
valued,  stationary,  bounded  and  right  continuous  stochastic 
process  and  define  MCt;)  =  v*Ct)  v' (t)  ,  M^(t)  =  MCt/e^),  M  =  EM(t)  , 
where  *  denotes  the  complex  conjugate.  Let  S  be  a  complex 
R  -valued  quantity  and  Gq,  G,  t  positive  real  numbers.  Let  WC*) 
and  W(*)  denote  the  solutions  to 

XW  +  (GM  +  I)W  =  GqS*  , 

x^^+  (GM^+  I)W^=  GqS*  ,  WCO)  =  W^(0)  . 

Such  equations  arise  in  the  study  of  adaptive  antenna  arrays  [14]. 
The  function  W(t)  converges  as  t  Define  x  (•)  =  (• ) 

-  W(*)3/e.  Then 

(6.1)  X  =  -  Y  (GM  +  I)x  -  Y  x  -  Y  ^  (0)  “  ®  » 

E  6  " 

where  6M  =  M  -  M.  The  asymptotic  properties  (large  t,  small  e) 
are  of  interest. 

O 

Define  the  operator  by 

00 

W'(t)  E5M'(s)f  (x)6M(0)W(t)ds  , 

0 

Let  6M(*)  satisfy  (E^  denotes  expectation  conditioned  on 
v(p) ,  p  <  t) 


O  n  ^ 

f(x)  =  — ^ 


'I 
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(6.2) 


-00 

|E^5M(t  +  s)|ds  ^  K,  all  u),t  , 
0  ^ 


drlE  6M(t 

J  0  ^ 


+ 


s)  6M’  (t+  s  +  r) 


-  EoMCt  +  s)6M’  (t  +  s  +  r)|<  K  , 


all  w,t 


Then  x^(/)  is  tight  and  converges  {12]  weakly  to  the  nonhomo - 
geneous  diffusion  x(.*)  with  generatorij^"  given  by 


(.6.3)  (|^  +  =  ■— -  f’(x,t)(GM  +  I)/T  +  A°f(x,t) 

As  t  ^  the  sequence  of  measures  y(t,*)  of  x(t)  converge 
weakly  to  the  invariant  measure  of  the  stationary  diffusion  which  is 

-  -  O 

obtained  when  W(<»)  replaces  W(t)  in  the  definition  of  A^.  De¬ 
fine  V(x)  =  x'x.  Then  Theorem  3  holds  with  the  condition  (6.2) 
replacing  the  mixing  condition.  The  proof  goes  through  owing  to 
the  fact  that  6M^(*)  appears  linearly  in  (6.1). 

Example  2.  Weak  convergence  of  (6.4)  was  investigated  in  [13]. 

x^=  H(x^)  +  Dy^ 

(6.4) 

y^=  LgQ^(s  +  n^-  J(x^))/e  ,  x^(0)  =  Xq  . 

In  (6.4),  gm(*)  is  an  approximation  to  a  "hard  limiter;"  a  func¬ 
tion  g(»)  is  a  hard  limiter  if  g(u)  =  sign  u,u  ^  0.  Such 
functions  occur  frequently  in  applications  in  control  and  communi¬ 
cations.  Instead  of  working  with  g(*)  directly,  we  worked  in 


J 
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[13]  with  the  approximation  where  g^jCO)  =  0,  sign  Cu ) 

=  signu,  g^(u)  =  sign  u  for  |u  |  >  a,  |g^Cu)f^l  and  ^Cu]  <  K/a 

for  I  u|  <,  «.  The  process  s(*)  is  a  bounded  right  continuous 

signal  which  is  a  Feller-Markov  (and  ergodic)  process  with  a 

£  2 

homogeneous  transition  function.  Let  n  Ct)  =  zCt/e  ]/e,  where 
z(’)  is  a  scalar  valued  Gaussian  process  with  correlation  function 

2  “  3.  t 

c  e  ,  a  >  0.  With  H(*]  continuous  and  J(*)  twice  continuously 

£ 

differentiable,  {x  CO)  is  tight  and  converges  weakly  Cas  e  -*■  0 , 
a  ^  0  such  that  ^  0)  to  the  x(«)  of  (6.5),  provided  that 

(6.5)  has  a  unique  solution  on  [0,")  (in  the  weak  sense)  for  each 
initial  condition.  Now,  assume  this  uniqueness.  Assume,  in  addi¬ 
tion,  that  J(0  and  its  partial  derivatives  up  to  order  2  are 
bounded  and  continuous,  and  that  H(x)  =  0(|xl)  for  large  lx]. 

Let  V(x)  =  x'Qx  for  Q  >  0  and  suppose  that  H(x)  <  -  YV(x) 

+  K  for  some  Y  >  0. 

dx  =  H(x)dt  +  LD  .C.s„:_^JC.O)  ^jt  +  LD  dB  , 

(6.5) 

xCO)  =  Xg  ,  B(*)  =  standard  Brownian  motion. 

£ 

The  perturbed  Liapunov  function  Vj^(*)  required  for  this  case 
can  be  constructed  in  a  very  similar  way  to  that  given  in  Theorem  3. 

g  N 

See,  e.g.,  the  way  the  analogous  f  ’  is  constructed  in  [13], 

We  then  get  the  situation  of  (5.1),  where  V(x)  and  V(x)  are 
0(|x|),  and  %  (t),  ^  (t)  can  each  be  taken  to  be  of  the  form 
lz(t/E^)I^.  The  0(e)  in  (.5.1)  need  to  be  replaced  by  OCe**’/^^). 

But  if  we  let  a  0  as  e  -►  0  such  that  0,  Theorem  4 

remains  valid  if  x(*)  satisfies  the  conditions  of  that  theorem. 


-26- 


This  example  is  a  simple  form  of  a  large  class  that  occurs 
naturally  in  control  and  communication  theory,  where  the  noise 
is  unbounded,  the  dynamics  non-linear  and  to  which  our  method 
can  be  applied.  The  general  ideas  of  Theorems  3  and  4  remain 
valid,  but  the  actual  details  of  proof  might  depend  on  the  spe¬ 
cial  conditions  of  the  application. 


J 


Discrete  Parameter  State  Dependent  Noise  Case. 


The  method  of  Theorems  1,  3  and  4  is  readily  extended  to  the 
discrete  parameter  case,  and  we  will  outline  the  idea  for  a  case 
where  the  noise  depends  on  the  state.  See  [9],  [10]  for  examples 
of  such  a  case.  Define 


(7.1) 


X" 


n  +  1 


= 


n 


eCCX^) 


6C(X®,5^)  ./e  FfX^.S^)  ♦  0(e) 


where  ^  bounded  noise  process.  In  many  important  cases, 

(5^}  depends  on  ([10])*  We  model  this  in  the  following 

way.  Let  ^  homogeneous  Markov  process  with  the  one 

step  marginal  transition  function  written  as 


p^C5.i,r|x)  -  Piil  crlxj  =  x.il  =  5)  . 

£ 

For  each  x,  define  the  bounded  chain  {5  (x) }  by  the  transition 
function  defined  recursively  by 


P^(C,  j,r|x) 


p^(C,j  -  ii,dCJx)  P^(Cj,Jl,r|x) 


£  •  £  “  £ 
Let  {C  (x) }  have  a  unique  invariant  measure  P  (•)  and  let  E 

denote  expectation  of  functionals  of  the  chain  under  the  station¬ 
ary  measure. 

The  comments  below  are  formal,  and  it  is  implicitly  assumed 
that  the  indicated  derivatives  and  sums  exist.  Suppose  that  there 
are  continuous  a^^(*)  and  F(*)  such  that  for  each  x  the 
limits  (7.2)  and  (7.3)  exist  for  smooth  f(*)*  (The  idea  is  sim¬ 
pler  for  the  case  of  non-state  dependent  noise,  but  owing  to  the 
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numerous  applications,  it  is  worthwhile  to  present  the  more  general 
case. ) 


(7.2) 


(7.3) 


OO 

I  E^[E(F(x,cJ(x))!Cq(x))1^  F(x,5q(x))  - 
^  “  1 

^  E^F*(x,Cq(x))  f^^(x)  F(x,Co(x)) 

00 


+  I  e'^F- (x,5^(x))  f^^(x)  F(x,;o(x)) 
^  1 


l.J  1  J 


E(x) 


Define  by 


(7,4) 


1.) 


3x .  3x^. 


*  I  ifi(x) 


£  £ 

Let  X  (•)  denote  the  niecewise  constant  process  with  values  on 

£ 

[en,en+e).  Under  some  additional  conditions,  {x  (•)}  converges 
weakly  to  the  diffusion  x(*)  with  generator  i?^[9,10]. 

For  the  discrete  case,  the  operator  is  defined  on  the 

set  of  functions  ^  which  are  constant  on  [ne,ne  +  e) ,  and 
{Xj,j  £  n,5j,j  <  n}  5  measurable  at  ne.  For  f(*)E-^,  A^f  is 

defined  by  A^f(t)  =  [E^f(t  +e)  -  f(t)]/e. 

Given  a  Liapunov  function  V(*),  the  perturbed  and  truncated 
V^(*)  are  found  by  a  method  that  is  very  similar  to  that  used  in 
Theorem  3,  but  with  summation  replacing  integration,  and  a  trunca¬ 
ted  Taylor  series  expansion  rather  than  a  differentiation  used  to 
£  e 

get  the  A  i^*^*  Perturbed  test  functions  are  constructed  in 
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[9] ,  110]  and  are  used  in  the  proof  of  the  weak  convergence  men¬ 
tioned  above  (following  the  method  of  [S]).  The  .(•)  would 
be  constructed  just  as  these  perturbed  test  functions  were,  but 
using  test  function  Vj^CO*  If  the  V^(*)  -  VC*)  and  A^V^C*) 

-  i^VCO  satisfy  the  bounds  (4.6),  (4.8)  in  the  set  then  the  dis¬ 

crete  parameter  analog  of  Theorem  3  holds.  Theorem  1  holds  in  any 
case  if  the  {x^(*)}  of  this  section  satisfies  the  conditions  of 


that  theorem. 


%— 
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